Papers with language model loss
APGN: Adversarial and Parameter Generation Networks for Multi-Source Cross-Domain Dependency Parsing (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for dependency parsing use labeled training data for several fixed domains, but performance drops when labeles only exist for several out-domains. |
| Approach: | They propose a model for multi-source cross-domain dependency parsing that uses a parameter generation network and adversarial network for learning domain-invariant representations. |
| Outcome: | The proposed model improves cross-domain parsing performance by about 2 points over strong BERT-enhanced baselines over a recently released dataset for multi-domain dependency parse. |
Robust Conversational Agents against Imperceptible Toxicity Triggers (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing work to generate adversarial attacks is costly and not scalable . despite the abundance of research in this area, little attention has been given to adversarials . |
| Approach: | They propose an adversarial attack mechanism that mitigates toxic language generation . they propose a defense mechanism that is scalable and can be generalized . |
| Outcome: | The proposed defense is effective at avoiding toxic language generation even against imperceptible toxicity triggers while preserving conversational flow. |
Semi-supervised Domain Adaptation for Dependency Parsing via Improved Contextualized Word Representations (2020.coling-main)
Copied to clipboard
| Challenge: | Recent advances in deep neural network models have improved parsing performance on in-domain texts . however, the problem is to improve performance on out-of-domain text data when there is only a small-scale out-domain labeled data. |
| Approach: | They propose to use adversarial learning and fine-tuning BERT to improve contextualized word representations on out-of-domain texts. |
| Outcome: | The proposed models achieve consistent improvement and fine-tune BERT processes boost parsing accuracy by a large margin. |
Translating away Translationese without Parallel Data (2023.emnlp-main)
Copied to clipboard
| Challenge: | Translated texts exhibit systematic linguistic differences compared to original texts in the same language, referred to as translationese . studies show translationeses have effects on various cross-lingual natural language processing tasks . |
| Approach: | They propose a translation-based style transfer approach that learns from monolingual data . they combine a self-supervised approach with an unsupervised approach . |
| Outcome: | The proposed method reduces translationese to a level of a random classifier after style transfer while preserving the content and fluency in the target original style. |